SBNews: News Robot Utility
(C) Scott M Baker
Note: This program is still in the early beta stages. It hasn't been
very extensively tested and it's very possible that you may experience trouble.
Please report any problems to me at smbaker@primenet.com.
I variously refer to this program under the names "NewsBot" and "SBNews"
throughout the document. You can for the most part consider them to be the
same. Eventually, I plan on including extra utilities in this package, "SBNews"
will be the name of the full package and "NewsBot" will be the name of the
News Robot utility within the package.
Purpose:
The purpose of this program is to automatically
download and uudecode files from newsgroups
via a winsock connection. If you've found this program,
then you probably know what this means, so I won't go into detail. Newsbot
will attempt to piece together multi-part files.
If you're new to binary newsgroups, then here are a few G-rated newsgroups
that you might wish to try:
alt.binaries.pictures.astro
alt.binaries.pictures.animals
alt.binaries.pictures.cartoons
alt.binaries.pictures.fractals
There are a wealth of adult-oriented newsgroups, but I'm not going to list them in this document.
I have included a sample file, SAMPLE_G.TXT, which includes a listing of binary newsgroups. There are no descriptions, but you can probably find out from the newsgroup name what subject matter the newsgroup deals with.
Requirements:
- Windows 3.1 or greater
- Winsock connection capability (i.e. SLIP or PPP)
Distribution:
- 16-bit (Windows 3.1) Version: SBNEWSxx.ZIP (xx denotes version
number)
- 32-bit (Windows 95) Version: SBN32_xx.ZIP (xx denotes version number
Features:
- Unattended download - do other work (or go out to lunch) while
downloads and uudecoding are all processed for you!
- Automatic HTTP address logging -- NewsBot will keep track of any HTTP
URL's that were found while downloading addresses. These will be saved to
a convenient HTM file, HTTPGRAB.HTM.
- Built in JPEG viewer. Allows you to see postage-stamp size images of
the files you are downloading in progress.
- Delete/View previously downloaded JPEG files.
- Comprehensive logging capabilities - save the subject and from information
for files that you might want to follow up on later.
- Dupe Checkers -- avoid downloading the same files multiple times.
Files Included:
- NEWSBOT.EXE: Executable; what you need to run to get the program
to work.
- NEWSBOT.HTM: Documentation in HTML format (read with your browser)
- NEWSBOT.DOC: Documentation in plain-ASCII format (read with any plain
ascii text editor)
- BWCC.DLL (16-bit) or BWCC32.DLL (32-bit): Borlands Custom-Control DLL
library; used to support Newsbot's grey dialog boxes.
- FILE_ID.DIZ: Brief description file; useful for bbs distribution
- SAMPLE_G.TXT: Plain-ASCII listing of binary newsgroups
Installing and Using the Program:
Here are some very quick notes on installing and using Newsbot:
Installation:
- Unzip the distribution archive into the directory of your choice.
Running:
- Run NEWSBOT.EXE -- this can be done in various ways - from the
Win-95 command prompt, the Windows Explorer (and/or file manager), the Windows
Run Program command, etc.
- On your first session, you'll be prompted for a "News Host Name". This
is the name of your news host, and should be given to you by your ISP (internet
service provider).
- On your first session, you'll be prompted for a "download path". This
is the path where you want the files you download to be placed.
- Once NewsBot is loaded, you'll want to add some newsgroups. This is done
with the "Add" button. You'll need to know the names of the groups you want;
you might want to consult SAMPLE_G.TXT for some sample newsgroup names.
- Press the "CONNECT" button to start everything. All else should be mostly
automatic from here on. Feel free to tinker around with the menu items
Command Line Parameters
- "-i INI_FILE_NAME". Sets the name of the INI file
to use. By default, newsbot uses the file newsbot.ini in your windows directory.
Specifying a filename here will cause newsbot to use the file under that
name in newsbot's home directory. For example "Newsbot -i
news1.ini" would use the filename "news1.ini" in newsbot's home directory.
Jpeg Viewer
SBNews has a built in Jpeg viewer for viewing JPEG files offline.
The JPEG viewer can be operated in several ways:
- By double-clicking on a [RECEIVED] entry in Newsbot's log window
- By single-clicking on one of the preview pictures on the right hand side
of Newsbot's main window.
- By using the FILE:VIEW JPEG option from Newsbot's pulldown menu.
Once loaded, the JPEG viewer window will display the JPG file selected
and include a listing of filenames and directories on the left-hand side
of the window. This listing is provided to let you easilly select other files
to view. Double clicking on a directory in the directory list will change
to that directory and double clicking on a filename in the file list will
display that file. The CHDIR and VIEW buttons perform the same functions,
respectively.
The "<" and ">" buttons will view the immediately previous and next
files in the currently selected directory. You may use these to rapidly view
a list of files that were downloaded overnight for example. The ">" button
will activate a slideshow of the current directory. Each file will be displayed,
there will be a slight delay, and the next file will be displayed.
The FILE menu of the viewer has several options:
- Open File: Allows you to open a new file using the windows common-dialog
open method.
- Delete File: Deletes the current file that you are viewing.
- Save as BMP: Saves the current image as a BMP image, which may later
be used as windows wallpaper, for example.
- Set as Wallpaper: Saves the current file as a BMP and tells window to
use it was wallpaper.
The SlideShow menu has a few options as well:
- Start/Stop: Starts and stops the slideshow, equivalent to the ">"
button.
- Set Interval: Sets the amount of delay between slideshow pictures.
- "Normal Mode": Images will be displayed in the jpeg viewers window.
- "Wallpaper Mode": Images will also be set as WallPaper. This sort-of
turns your entire desktop into a slideshow display. You can minimize the
JPEG viewer window and continue to do work while images display in the background
as wallpaper.
Configuration
Host Name:
The host name is the name of the nntp host which NewsBot will connect
to. If you don't already know this, then you may wish to contact your Internet
Service Provider (ISP) or check an existing news program on your system for
the name.
Download Path:
The base path where downloaded files will be placed. If you enable
any of the download path expansion (see misc. preferences), then files may
be downloaded into sub-directories of this base path.
Miscellanious Preferences:
Miscellanious preference options are located under the Configure
menu under the heading Preferences.
- Logging: Logging options control the creation of
the newsbot.log
file. The log may be useful so that you have more information about the files
that you have downloaded.
- Log Enabled: if checked, then a log file will be
written. Otherwise, no log will be written.
- From user name: records the name of the person who posted
the message
- Subject: records the subject line of the message
- Comments: records the first few non-file lines of the
message. Useful if the sender has prefaced the message with some explanation
of it's content.
- Too Small: Messages that are below the minimum message
line limit (see below).
- Too Long: Messages that are above the maximum message
line limit (see below).
- Dupe File: Files caught by the dupe-file checker.
- Dupe Subject: Messages caught by the dupe-subject checker.
- Mime Headers: Really just debugging info for me....
- Path Expansion: These options will place files
from different groups
in different download directories.
- None: Places all downloaded files in the same download
directory.
- Group Number: Places files in directories designated
by their group numbers in the group list (i.e. the first group would go in
.../group1/, the second in .../group2/, etc)
- Group Name: Transforms the group name into a download
path. For example, alt.binaries.pictures.astro would go into
.../alt/binaries/pictures/astro/.
- Case Conversion: These options will convert the
case of the filename
to a uniform format. Due to limitations in the 16-bit application model,
filenames will probably be always upper case in the 16-bit executable.
- None: Leave filename case as it appears over the
modem.
- Lower: Convert all filenames to lower case.
- Upper: Convert all filenames to upper case.
- Disconnect/Reconnect: The disconnect/reconnect
system will automatically disconnect and reconnect the current nntp session
in order to abort a message that is being transmitted. This is used to save
time, rather than downloading entire messages which are not stored on disk.
If you disable disconnect/reconnect, then Newsbot will receive the entire
message, although the message will not be written to disk. You may toggle
disconnect/reconnect on or off for several subcases:
- No-Data: Messages with more non-encoded lines than
the no-data threshold set below.
- Dupe-File: Files caught by the dupe-file checker (not
relavant to the dupe-subject checker, as the dupe-subject checker won't download
the message in the first place)
- No-Mask: Files which do not match an acceptable file
mask.
- Current-Delete: Files which were deleted during transfer
by a user request.
- Message Line Limits: Newsbot can limit which messages
are downloaded
based on the length (number of lines) of the message.
- Minimum Lines: Messages with fewer than the specified
number of lines will not be downloaded. The rationale is that small messages
do not contain any useful information (pictures are big!) and some efficiency
may be obtained by not downloaded them.
- Maximum Lines: Messages with greater than the specified
number of lines will not be downloaded. The rationale is that really huge
messages are a waste of time. It's usually the case of someone who scanned
in a picture at too fine a resolution.
- "No Data" Threshold: This specifies the number of lines
that can be in a message when no attached files (eg images) can be found
before the message is skipped. For example, if 500 lines are scanned, and
no attached image is present, then the message is probably grunged, a misplaced
part of a multipart encode, or some other useless data.
Lockout Poster List
The lockout poster list is a listing of email names of people from
whom you DO NOT wish to receive news articles from. The customary way to
add new posters is to notice an image you don't like in the JPEG Preview
and click the Lockout Poster option under the Current or Previous menus.
Authentication:
The authentication options are used for news servers which require
a user name and password to access the newsgroups. If your news server does
not require a name and password, then you should leave this option alone.
Authentication is located under the Configure menu.
Acceptable File Masks
Newsbot allows you to specify which types of files will be downloaded.
Specifying "*.*" will enable any file to be downloaded, and this is the default.
For example, if you only wanted to receive images, then you may wish to remove
*.* and add in *.gif and *.jpg. If you only wanted archives, add in *.zip.
etc.
Dupe Checking
Two different types of dupe checking are supported.
They may be used independantly, or you can use both of them at the same time
if you wish. By default, the dupe checkers maintain a list of approximately
the most recent 2048 messages received.
- Subject Dupe Checker: Remembers duplicates by
keeping
track of the subject and from user name of the message. Thus, if the same
person posts multiple messages with the same subject, the message will be
flagged as a dupe. This is useful for preventing download of crossposted
messages (i.e. identical messages posted in multiple groups)
- Filename Dupe Checker: Remembers duplicates by keeping
track of the names of the files that were downloaded. This is very effective
at weeding out dupes, but has the side effect of sometimes deleting messages
that are not really duplicates. For example, there are a lot of files name
"1.JPG" out there!
Both of the two above mentioned dupe checkers have some options that
control their behaviour:
- Reject Duplicates: If checked, then dupe
checking
will be performed as stated above. If not checked, then no dupe checking
will be performed.
- Save dupe list: If checked, then dupe information will
be saved from session to session. If not checked, then dupe information will
only be kept for the current session.
- Items to Keep: This is the number of items to "remember"
for the dupe checker. A bigger number here will remember more messages (or
files), but will also require more memory and/or disk space to hold to dupe
checker information.
Statistical Options
There are a variety of options listed under the statistics menu item.
Most of these simply return information that SBNews has collected while
processing newsgroups.
- Similar NewsGroups: Newsgroup headers contain an
"Xref:" line which lists other newsgroups to which a specific article has
been posted. Many times, a poster will "cross-post" an article to multiple
groups which share the same interests. The Similar Newsgroups list will display
all of the accumulated Xref's for the current area.
- HTTP Grabber: The HTTP Grabber automatically keeps a
list of any WWW addresses that are found in the text and/or subject lines
of downloaded messages. In addition, a HTML file called HTTPGRAB.HTM will
be written to the newsbot directory. You may load this file with your WWW
browser and look up any references that SBNews found.
- Dupe File List: The list that the dupe-checker keeps
in memory to catch dupe files.
- Dupe Subject List: The list that the dupe-checker keeps
in memory to catch duplicate subject lines.
Unfinished Stuff
This program is still largely unfinished. There's lots of
"bells-and-whistles" that I'm planning on adding in the future. So don't
dispair if a few features that you'd like are missing. Important things I'm
hoping to add are:
- Automatic Connect and Disconnect (i.e. start receiving when it's
loaded)
- Ability to download newsrc (the list of newsgroups)
- Ability to specify search filters (i.e. only retrieve messages with xxx
in the subject field)
- Manual toggling of articles you want to download
Registration
SBNews/Newsbot is a shareware program and as such, you are only granted
the right to operate it for a limited time to evaluate it's performance.
Continued usage requires registration in the amount of $15.00.
The registration policy is as follows:
- Mail $15 (a personal check made out to "Scott M. Baker" is fine)
to my US-Mail address. Include your EMAIL address with the check.
- I will generate a registration code that corresponds to the email
you included. I will email this registration code back to you at the address
you provided.
- Load up newsbot and select Register from the menu. Enter your
email and the code I provided. The program will now be registered.
- Your registration is good forever, and will be supported under ALL
FUTURE VERSIONS of this program.
Registration entitles you to use of the program (and all future upgrades)
for as long as you wish. The 15-second shareware delay while loading will
be removed.
In addition, if I add additional utility programs to the SBNews package,
your registration will entitle you to usage of them as well.
Please do consider registering this shareware if you continue to use it.
The programming is very involved and time consuming, and it really gives
me a lot more incentive to add new features if I receive compensation for
my work.
How to contact me:
US-Mail:
Scott M. Baker
2241 W Labriego
Tucson, Az 85741
My Bulletin board:
The Not-Yet-Named BBS
(520) 544-4655 (USR Dual 14.4k)
(520) 797-8573 (USR Sportster 28.8k)
Email:
smbaker@primenet.com
My Homepage:
http://www.primenet.com/~smbaker
Revision History
- Version 1.0
- Version 1.1
- Added about dialog and shareware delay
- Cleaned up (shrunk) main screen and reorganized information
- Added context-sensitive hint text at bottom
- Fixed problem with message pointers not being recorded properly
- Version 1.2
- Added JPG viewer window (supported in 800x600 or above only)
- Probably fixed a nasty stack overflow (Runtime Error 202) bug that was
causing trouble with new users.
- Version 1.3
- Made JPEG viewer more crash resistant
- Increased JPEG viewer read buffer to deal better with images with large
headers
- Added a title to the main window
- Finished registration code system
- Version 1.4
- Version 1.5
- Made sure to free gensock.dll on exit
- Added Configure:Preferences dialog, many new preference options
- Version 1.6
- Raised counter-over threshold from 250 to 1000
- Set update timer at 100ms instead of 250ms
- Added Lockout Poster system
- Revamped dupe checker for better reliability
- Added similar newsgroups list
- Raised maximum number of newsgroups from 32 to 128
- Added view and delete options to current and previous menus
- Fixed problem with crashes on high-ascii characters in lockout poster
list
- Fixed possible bug with long header lines
- Now handles nonexistant newsgroups without halting
- Version 1.7
- Shareware delay now increments by 1 every day Newsbot is used
- Newsbot now stops loading groups from INI file when first blank entry
is found (speedup?)
- Revamped the dupe message checker (again!)
- Fixed bug which was causing unnecessary JPEG error 117
- Fixed problem with decoding lines that started with a double period
- Version 1.8
- Added message when old BWCC found
- Version 1.9
- Change "about" menu to "file" menu
- Added open jpeg viewer option to file menu
- Prevented crash if 0 messages in a group
- Added HTTP URL Grabber
- Fixed problem with long subject/from lines (i.e. >256 chars)
- Double clicking on the log will bring up a viewer
- Made sure files are written to proper directory
- All uudecode filenames are normalized to lower case
- Added dupe file checker
- Revamped dupe subject checker (again!)
- Version 2.0
- Added authentication support (AUTHINFO Original)
- Ability to deal with SOME mime messages
- Added acceptable file mask dialog
- 32-bit version available
- Incorporated gensock code into SBNews; Gensock.DLL no longer needed.
- Fixed memory leak in jpeg viewer
- Used suballocater to deal with strman hash table nodes
- Added "add" and "refresh" options to similar groups dialog
- Version 2.1
- Spiffed up Jpeg viewer window
- Fixed up error handling on JPEG decoding routines
- Fixed possible error with "Null" strings
- Version 2.2
- Fixed problems with palette in 256 color mode
- More tolerant of uuencoded lines with extra characters
- Made small version of window even smaller
- Dupe File, Delete Current, and No Mask signals will now disconnect/reconnect
the session to keep from having to download entire message.
- Several user-friendliness improvements
- Included SAMPLE_G.TXT, a list of binary picture newsgroups
- Added "-i [INIFILENAME]" command line parameter
- Added Save as BMP option to JPEG viewer
- Added Set Wallpaper option to JPEG viewer
- Added normal vs Wallpaper mode toggle to JPEG viewer slideshow
- If you drag & drop a JPG file into newsbot's main window, then Newsbot
will launch a Jpeg viewer window to view the file.
- Moved dupe checker preferences to their own menu item
- Added "Items to keep" settings for dupe checkers
- Fixed major distribution problem in hash function
- Added Blank option to Current/Previous menus to blank out the current picture
- Viewer enable defaulted to active
- Fixed problem with long pathnames in jpeg viewer
- Fallback when improper BWCC.DLL loaded
- Version 2.3
- Fixed GPF on loadup of nonregistered users